\documentclass[a4paper,times]{cjeart}
%\usepackage{moreverb,url}

\usepackage[colorlinks=false]{hyperref}
% \usepackage{cite}
\usepackage{mdwmath}
\usepackage{mdwtab}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{subfig}
%% \usepackage{subfigure}
\usepackage{array}
\usepackage{multirow}
\usepackage{booktabs}
\usepackage{url}
\usepackage{algorithm,algorithmic}
% \usepackage{algpseudocode}
% \usepackage{algpascal}
\usepackage{epstopdf}
\usepackage{color}
\usepackage{bm}
\usepackage{amsthm} % 如果要使用proof语句就必须要引入这个包

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}
\newtheorem{lemma}{Lemma}
\newtheorem{definition}{Definition}

\renewcommand{\thefootnote}{\arabic{footnote}}


\volumeyear{2022}
\volumenumber{04}
\issuenumber{04}
\journalname{Chinese Journal of Electronics}
\startpage{1}
%\endpage{6}
\DOI{123.123.123.123}
\journaltype{Research Article}


\title[Encrypted Malware Traffic Detection via Multi-view Learning]{Encrypted Malware Traffic Detection via Multi-view Learning}

\author{%
Firstname1 Middlename1 Lastname1\affilnums{1,2,3}, 
Firstname2 Middlename2 Lastname2\affilnums{2}, 
Firstname3 Middlename3 Lastname3\affilnums{4}, 
}

\affiliation{%
\affilnum{1}Institution Name, City name and postal code, Country\\
\affilnum{2}Institution Name, City name and postal code, Country
}

\corrauth{Firstname2 Middlename2 Lastname2}
\email{xxxxxxxxxxx@123.com}

\receivetime{Received December 22, 2022}
\accepttime{Accepted xx xx, xxxx}
\publishtime{Published xx xx, xxxx}

\authorcopy{Firstname1 Middlename1 Lastname1\emph{et al.}}


\abstract{%
% With the encryption protocols widely adopted in network communication, traditional deep packet inspection methods fail on detecting malware traffic. 
% To address this issue, the emerging statistic approaches classify malware traffic by extracting features from observable data fields. 
% However, they ignore the feature redundancies and directly concatenate them into a unified vector which is further fed into classic statistic models, such as SVM and kNN.
% In this paper, we consider TLS encrypted malware traffic and propose a multi-view classification model to detect it. 
% In specific, twenty-one features are extracted and semantically separated into three sets, i.e. packet feature, TLS feature and certificate feature, with each corresponding to an independent view.
% Then, a purposive multi-view neural network model is designed to integrate complementary information from the three data views. 
% To the best of our knowledge, this is the first attempt to apply multi-view learning technique on malware traffic detection in literature, which would provide beneficial guidance to further research.
% Additionally, we conduct extensive experiments on the proposed model to verifies its effectiveness and superiority.
% The result shows at least xx.xx\% precision on CTU-13 and MCFP datasets. 
With encryption protocols widely adopted in network communication, traditional deep packet inspection methods frequently fail to detect malware traffic. 
Recent statistical approaches address this issue by classifying malware traffic based on features extracted from observable data fields. 
However, these approaches ignore potential redundancies in the features. 
Addressing this issue, we propose a multi-view classification approach to detect TLS encrypted malware traffic. 
It first extracts 21 features separated into three semantic sets/views (packet feature, TLS feature or certificate feature).
Then a multi-view neural network integrates the complementary information from the three views. 
To the best of our knowledge, ours is the first attempt to apply multi-view learning on malware traffic detection. 
We conduct extensive experiments demonstrating the method's efficacy.
}

\keywords{Encrypted traffic, Malware traffic detection, Multi-view classification, Neural networks.}


\begin{document}


\maketitle

\section{Introduction}
\label{sec1}

With the rapid advancement of information technology, network security is becoming a significant concern in people's life. 
To protect user privacy, encryption techniques (especially the well-known Transport Layer Security (TLS) protocol) have been widely adopted in network communication. 
% It is reported by Gartner that more than 80\% corporate traffic is encrypted in 2019.
% https://gbhackers.com/cisco-eta-encrypted-traffic/#:~:text=Cisco%20ETA%20%E2%80%93%20Provides%20Solution%20for%20Detecting%20Malware,June%20and%20now%20it%20came%20for%20general%20availability.
\begin{figure}[h] 
  \centering
  \includegraphics[width=0.85\linewidth]{fig/https-crop}
  % \caption{Percentage of web pages loaded by \emph{Chrome}\protect\footnotemark[1] browser using HTTPS across different operation systems. The corresponding data has been retrieved from \emph{Google Transparency Report}\protect\footnotemark[2].}
  \caption{Percentage of web pages loaded by \emph{Chrome}\protect\footnote[1]{\url{https://www.google.com/chrome/}} browser using HTTPS across different operation systems. The corresponding data has been retrieved from \emph{Google Transparency Report}\protect\footnote[2]{\url{https://transparencyreport.google.com}}.}
  \label{fig:https}
\end{figure}
In recent years, encrypted traffic has been increasing by more than $80\%$ annually (see Fig. \ref{fig:https}).
However, attackers can use TLS encryption to bypass traditional deep packet inspection (DPI) devices and deliver malicious payloads.
Detecting such malicious TLS traffic is a significant problem in computer security.

Some work provides solutions by adopting the DPI detection framework.
Most of them employ a Man-in-the-Middle (MitM) approach, where the encrypted traffic is interrupted, decrypted, and analyzed by a middlebox \cite{DBLP:conf/sigcomm/SherryLPR15,DBLP:journals/ton/FanGRCQ17,DBLP:conf/ccs/NingPLCC19,DBLP:conf/ccs/CanardDKPS17,DBLP:conf/nsdi/LanSPRL16,DBLP:conf/infocom/YuanWLW16}. However, these methods can cause both practical and privacy concerns \cite{DBLP:journals/corr/abs-2010-16388,DBLP:journals/corr/abs-2101-04338}. 
On the one hand, the middlebox introduces extra costs on appliances and latency in the encryption-decryption process. 
On the other hand, it violates the end-to-end privacy guarantee, resulting in potential information leakage.

Anderson et al. observed that the statistical features of traffic flow and the TLS protocol's observable data fields exhibit discriminative behaviors between normal and malware traffic \cite{DBLP:conf/ccs/AndersonM16}.
Based on this observation, They disassemble the building process of TLS connection and extract a large set of statistic features from traffic flow and TLS protocol parameters, such as packet time sequence, packet length, TLS version, etc. \cite{DBLP:conf/kdd/AndersonM17,DBLP:journals/virology/AndersonPM18}
Liu et al. select twenty-three potentially information-carrying features and separate them into three categories: packet feature, TLS feature, and certificate future \cite{DBLP:journals/cmc/Jiyuancmcmaldetect}.
In addition, they consider DNS request and response information \cite{david2018identifying}.

Compared with DPI-tailored methods, statistic solutions enjoy multiple merits. 
One of the most important is that there is no need to decrypt the TLS connection, ensuring the privacy guarantee and reducing the traffic latency.
Some of them can even give out the detection results before the exchange of application data, making it possible to detect malware traffic before it happens \cite{DBLP:journals/cmc/Jiyuancmcmaldetect}. 
However, current researches ignore redundancy among the extracted features and directly concatenate them into a unified vector to train the statistic models, such as kNN and SVM, resulting in unpromising detection rate.
To address this issue, we first extract three sets of features from TLS traffic flows by following \cite{DBLP:journals/cmc/Jiyuancmcmaldetect}.
Then, we propose a multi-view neural network model to classify whether a target flow is normal or generated by malware.
To the best of our knowledge, this is the first attempt to apply multi-view learning technique on malware traffic detection in literature, which would provide beneficial guidance to further research.
Additionally, we conduct extensive experiments on the proposed model to verifies its effectiveness and superiority.
% Corresponding malware traffic detection framework is visualized in Fig. \ref{fig:overview}.
% In addition, extensive experiments are conducted on CTU-13 and MCFP datasets to validate effectiveness of the proposal.
% Overall, the contributions are summarized as follows:
% \begin{enumerate}
%   \item 
% \end{enumerate}


\section{Related work}

In this paper, two research fields are concerned, including encrypted malware traffic detection and multi-view learning.
Therefore, we respectively introduce them in brief as follows.

\subsection{Encrypted malware traffic detection}

With more and more malware adopting TLS encryption techniques to secure their network activities, security community develops a large volume of methods to classify the target flow is normal or malicious.
Current researches can be roughly grouped into two categories, i.e. DPI-tailored and statistic approaches.

By adopting MitM style, DPI-tailored methods first decrypt the traffic, then analyze the inside details, at last encrypt and re-transmit it to clients.
MitMProxy\footnote[3]{\url{https://mitmproxy.org}} and SSLSplit\footnote[4]{\url{https://www.roe.ch/SSLsplit}} are two widely used open-source tools. 
For a client-server communication, client first builds a TLS connection with the middle box and sends traffic packets. 
When middle box receives the packets, it decrypts, analyzes, re-encrypts and forwards them to server with a new TLS connection.
Some researches propose to share TLS parameters with the middle box, which significantly reduces the latency of duplicate connections \cite{DBLP:journals/ccr/NaylorSVLBLPRS15,DBLP:conf/conext/NaylorLGKS17,DBLP:conf/sp/BhargavanBDFO18,DBLP:conf/ndss/LeeSLCCCK19}.  
However, extra costs on appliances and latency in the encryption-decryption process are also non-negligible due to the large-scale and high-speed properties of network traffic.
To ease these issues, some researchers propose to directly perform packet inspection on the encrypted data without decryption, such as BlindBox \cite{DBLP:conf/sigcomm/SherryLPR15}, SPABox \cite{DBLP:journals/ton/FanGRCQ17}, BlindIDS\cite{DBLP:conf/ccs/CanardDKPS17} and PrivBox \cite{DBLP:conf/ccs/NingPLCC19}.
Taking BlindBox \cite{DBLP:conf/sigcomm/SherryLPR15} for an instance, a set of attack rules are first generated by trusted organizations, such \emph{McAfee}\footnote[5]{\url{https://www.mcafee.com}} and \emph{Avast}\footnote[6]{\url{https://www.avast.com}}.
At the beginning of information exchange between clients, keywords in attack rules are encrypted and transmitted along with application data.
Once the middle box observes a new connection, it searches the encrypted keywords in encrypted application data according to attack rules.
Furthermore, Lai et al. improve the keyword matching algorithm to secure the middle box \cite{DBLP:journals/iacr/LaiYSLSSL20}, while Ning et al. propose to protect the privacy of attack rules, since they are the properties of trusted organizations \cite{DBLP:conf/esorics/NingHPXLWD20}.
However, one drawback is observed in aforementioned methods that the middle box is allowed to decrypt the data for further detection, if a connection is judged as suspicious by keyword matching.
This still violates the end-to-end privacy guarantee of traffic encryption protocols.

To maximumly preserve the security and privacy, the emerging statistic approaches passively analyze the traffic without interrupting the TLS connection directly.
It is observed that normal and malware traffic shows discriminative behaviors on observable meta data which can be quantized into feature vectors.
For example, Anderson et al. extract a large set of features, such as packet length/time sequence and TLS version, to classify if a traffic flow is malicious \cite{DBLP:conf/ccs/AndersonM16}.
They also find statistic model is slow to materialize in the network security domain due to inaccurate ground truth and non-stationary data distribution, and solve them in \cite{DBLP:conf/kdd/AndersonM17}.
Nevertheless, four feature sets, including flow metadata, packet length/time sequence, byte distribution and unencrypted TLS header information, are defined in \cite{DBLP:journals/virology/AndersonPM18}.  
These researches make it possible to detect the encrypted malware traffic without decrypting it. 
Liu et al. further select twenty-three features from connection building process, enabling the built model to identify security threatens before the malicious behaviors happens \cite{DBLP:journals/cmc/Jiyuancmcmaldetect}.
In addition, they employ online random forest model to adapt to rapidly changing malware traffic.


\subsection{Multi-view classification}

Multi-view learning aims to improve the model performance by integrating complementary information beneath each data view. 
According to the absence of supervisory signals, current literatures can be categorized into three groups, i.e. multi-view clustering \cite{DBLP:conf/aaai/0003LWZY21,DBLP:journals/tkde/JiyuanTKDE20,9399655}, semi-supervised multi-view learning \cite{DBLP:conf/aaai/NieCL17,DBLP:conf/acml/BoKZSC19}, multi-view classification \cite{DBLP:journals/corr/abs-2102-02051,DBLP:journals/corr/abs-2011-06170,DBLP:journals/pami/HuLT18}.
In this paper, multi-view classification is concerned, since it outperforms the others by large margins in most real-world applications.
% related work
With the wide spread of neural networks, the emerging deep multi-view classification catches the eyes of researchers.
Kan et al. develop MvDN network composed by view-specific and common sub-networks, where the former removes the discrepancy among views, while the later obtains the common representation shared by all views \cite{DBLP:conf/cvpr/KanSC16}. 
Hu et al. jointly learn an optimal combination of multiple distance metrics on view-specific and common latent representations \cite{DBLP:conf/sigir/HuZPL19}.
Instead of adopting the common auto-encoder structure, Zhang et al. concurrently reconstruct all data views and labels from a consensus representation which is further optimized along with network parameters \cite{DBLP:journals/corr/abs-2011-06170}.
Han et al. provide a multi-view classification paradigm to give out corresponding reliabilities along with predictions via employing Dirichlet distribution to model class probabilities and integrating evidence from each view \cite{DBLP:journals/corr/abs-2102-02051}.


\section{The propose method}

Compared with DPI-tailored methods, statistic solutions enjoy considerable merits. 
Significantly, there is no need to decrypt the TLS connection, ensuring the privacy guarantee and reducing the traffic latency.
Some statistical approaches can even give out the detection results before the exchange of application data, making it possible to detect malware traffic before it happens \cite{DBLP:journals/cmc/Jiyuancmcmaldetect}. 
However, existing approaches simply concatenate the features and then train a statistical learning method (such as $k$-nearest neighbors or a support vector machine). \emph{This ignores the potential redundancies in the extracted features}. In our experiments (shown in Section~\ref{sec:exp}), we find that this may result in a drop in detection rate.
We propose a new methodology addressing this issue. It first extracts three sets of features from TLS traffic flows similar as in \cite{DBLP:journals/cmc/Jiyuancmcmaldetect}.
Then, we use a multi-view neural network model to classify whether a target flow is normal or generated by malware.
To the best of our knowledge, this is the first attempt to apply multi-view learning to malware traffic detection. 
Our extensive experiments show the effectiveness and superiority of the proposed approach.

\begin{figure}[h] 
  \centering
  \includegraphics[width=0.8\linewidth]{fig/tls_flow-crop}
  \caption{A typical TLS connection between client and server.}
  \label{fig:tls_flow}
\end{figure}

\begin{figure*}[t] 
  \centering
  \includegraphics[width=0.95\linewidth]{fig/overview-crop}
  \caption{Overview of the proposed encrypted malware traffic detection method. Two independent parts are concerned, including feature extraction and multi-view classification. 
  % Note that, the feature extraction operates in bypass mode. Meanwhile, the extracted features can be semantically separated into three groups, i.e. packet, TLS and certificate features.
  }
  \label{fig:overview}
\end{figure*}

\subsection{Feature extraction}  

Similar to most researches, the proposed method is developed to detect encrypted malware traffic on flow (connection) level.
Fig. \ref{fig:tls_flow} shows a typical TLS traffic flow, where the device requiring to connect is defined as client and the other as server. 
At the beginning, three-way TCP connection is built, since TLS protocol operates on it.
Next, TLS connection is constructed by exchanging parameters between client and server.
For example, In \emph{Client Hello} message, client tells server which version of TLS protocol is preferred during the upcoming \emph{Data Exchange} process.
Once all parameters are discussed and agreed, application data is going to be transmitted.
Note that, we mark the target flow with five tuples, i.e. $<$\emph{client IP}, \emph{client port}, \emph{sever IP}, \emph{server port}, \emph{protocol}$>$, where $<$\emph{server IP}, \emph{server port}, \emph{client IP}, \emph{client port}, \emph{protocol}$>$ refers to the same flow as well.
Nevertheless, we adopt 
% the open-source tool \emph{MalDetect}\footnote[7]{\url{https://github.com/liujiyuan13/MalDetect}} to 
capture the traffic in bypass mode, which ensures the privacy guarantee and reduces the traffic latency compared with middle box.

Instead of directly feeding the raw data into the following statistic model \cite{DBLP:journals/iotj/ShafiqTBDG21}, we propose to extract three sets of hand-craft features from the aforementioned traffic flow. 
The used packets are captured from TCP and TLS building process rather than data exchange process, enabling the proposed model to be capable of classifying whether a TLS connection is normal or generated by malware before malicious behaviors happen.
Specifically, we follow \cite{DBLP:journals/cmc/Jiyuancmcmaldetect} and define three sets of features in the following.
At the beginning, packet features are described as:
\begin{enumerate}
  \item \emph{Inbound Bytes}: Sum of flow bytes sent from server to client.
  \item \emph{Outbound Bytes}: Sum of flow bytes sent from client to server.
  \item \emph{Inbound Packets}: Number of packets sent from server to client.
  \item \emph{Outbound Packets}: Number of packets sent from client to server.
  \item \emph{Duration}: Length of capturing time. 
  \item \emph{SPL}: Sequence of packet length. We discretize packet length into a $11$-bin vector with step size $150$ by following Lemma \ref{lemma:sequence}.
  \item \emph{SPT}: Sequence of packet time. We discretize time interval into a $11$-bin vector with step size $50ms$ by following Lemma \ref{lemma:sequence}.
\end{enumerate}
\begin{lemma}\label{lemma:sequence}
  For a set of numbers $\{x_i\}_{i=1}^n$, corresponding $m$-bin vector $\mathbf{b}\in\mathbb{R}^m$ of step size $s$ is defined as 
  \begin{equation}
    \mathbf{b}_j = |\mathcal{S}_j|,
  \end{equation}
  where 
  \begin{equation}
    \mathcal{S}_j = \left\{i\;|\; j\leq x_i < j*s, \forall j\in \{1,2,\cdots,m\} \right\}
  \end{equation}
  and $|\cdot|$ refers to the length of set.
\end{lemma}
Nevertheless, we extract six TLS features from the observable fields in TLS building process.
They are
\begin{enumerate}
  \item \emph{TLS Version}: In \emph{Client Hello} packet, client assigns the TLS version in the following connection. Four main versions are concerned in this paper, including SSL 3.0, TLS 1.0, TLS 1.1 and TLS 1.2.
  % \item \emph{Offered Cipher Suites}: In \emph{Client Hello} packet, client offers a list of cipher suits supported by itself to server. 
  % \item \emph{Selected Cipher Suite}: Once receiving the \emph{Offered Cipher Suites} in \emph{Client Hello} packet, server select one and response in \emph{Server Hello} packet.
  \item \emph{Offered Compression Method}: In \emph{Client Hello} packet, client offers a list of compression methods supported by itself to server. 
  \item \emph{Selected Compression Method}: In \emph{Server Hello} packet, server selects a compression method for further connection.
  \item \emph{Offered Extensions}: Extensions are used to provide more TLS details, such \emph{Server Name}. The extension type is considered. 
  \item \emph{Selected Extensions}: Server responses client some of extensions in \emph{Server Hello} packet. 
  \item \emph{TLS Packet Ratio}: During the capturing period, both TLS and non-TLS packets are transmitted between client and server. Thus, TLS ratio is considered.
\end{enumerate}
Additionally, X.509 certificate is an essential component in TLS connection. 
Most malware adopt self-signed certificates, while normal applications use authorized ones.
We extract eight features as follows:
\begin{enumerate}
  \item \emph{Certificate Number}: In most cases, one or more certificates are employed in a TLS connection.
  \item \emph{Bad Certificate Number}: Some certificates are formatted in a mess. 
  \item \emph{Version Ratio}: The ratio of a specific certificate version.
  \item \emph{Extension Ratio}: Multiple extensions are present in a certificate. Thus, the ratio of different extension type is considered.
  \item \emph{Validity Mean}: Every certificate is produced with a validity period. Their mean is considered.
  \item \emph{Public Key Length Mean}: Public key length mean of all certificates is considered. 
  \item \emph{Public Key Algorithm Ratio}: Public key algorithm is used to generate the public key in certificate. Their ratio is considered.
  \item \emph{Signature Algorithm Ratio}: There is a field in certificate to keep signature of the issuer. The ratio of corresponding signature algorithm is considered.
\end{enumerate}
Overall, we extract seven packet features, six TLS features and eight certificate features, respectively.

\begin{table*}[t]
\centering
\caption{Details of selected normal and malware traffic.}
\small
\label{tab:traffic_detail}
\begin{tabular}{llcl}
\toprule
Abbreviation & Identified Name & Flow Number & Description \\ \midrule
Normal & - & 66635 & Generated by non-malware or background traffic\\
Adw & Win32: Adware-gen & 29762 & Download and install other threats when run into the computer \\
Drp & Win32: Dropper-Gen & 358247 & Charge up even more autonomous malware, such as Trojans, worms and backdoors\\
Rtk & Win32: Rootkit-gen & 30592 & Target at the core system of Windows by executing a series of commands\\
Susp & Win32: Evo-gen & 35118 & Gather data, such as system settings, and send to remote attacker for analysis\\
\bottomrule
\end{tabular}
\end{table*}


\subsection{Multi-view classification}

To our best knowledge, current statistic methods to detect encrypted malware traffic concatenate the extracted features into a unified vector directly. 
However, different features are not comparable, especially those of different categories. 
As for the aforementioned three sets of features, their redundancy is another concern in statistic model training.  
We treat each feature set as a dependent data view of traffic flows.
Therefore, a multi-view classification neural network model is proposed for malware traffic detection.

Let $\mathcal{X} = \{\mathbf{x}_i^{(v)} \in \mathbb{R}^{d_v}\}_{i,v=1}^{n,V}$ be $n$ data entries with $V$ views and $\mathbf{Y} \in \{1,2,\cdots,K\}^{n\times 1}$ be corresponding ground truth, where $d_v$ and $K$ refers to the dimension of $v$-th view and class number, respectively.
Multi-view methods aim to improve the classification performance by integrating the complementary information among different views.
In this paper, a fully-connected neural networks model is adopted.
As shown in Fig. \ref{fig:overview}, the raw feature vectors are first mapped into latent representations, then fused, and finally compute the result with a consensus classification network.

Defining the encoder network corresponding to $v$-th view as $f_v(\cdot)$, corresponding latent representations $\{\mathbf{h}_i^{(v)}\}_{v=1}^V$ can be obtained as
\begin{equation}
  \mathbf{h}_v = f_v(\mathbf{x}_i^{v}),
\end{equation}
in which $f_v(\cdot)$ is initialized by auto-encoder pre-training.
Then, the consensus latent representation $\mathbf{h}_i$, as shown in Fig. \ref{fig:overview}, is obtained by concatenating $V$ view-specific ones and the fused $\mathbf{h}_i^{(c)}$ is computed via 
\begin{equation}\label{eq:latent_rep_v}
  \mathbf{h}_i = [\mathbf{h}_i^{(1)}; \mathbf{h}_i^{(2)}; \cdots; \mathbf{h}_i^{(V)}; \mathbf{h}_i^{(c)}], 
\end{equation}
where $[\cdot;\cdot]$ represents the horizontal concatenation and
\begin{equation}\label{eq:h_c}
  \mathbf{h}_i^{(c)} = \mathcal{M}(\mathbf{h}_i^{(1)}, \mathbf{h}_i^{(2)}, \cdots, \mathbf{h}_i^{(V)}).
\end{equation}
Note that, $\mathcal{M}(\cdot)$ is the multiplexer to fuse $\{\mathbf{h}_v\}_{v=1}^V$ and instanced in multiple ways in literature \cite{DBLP:journals/pami/HuLT18}.
Here, 
\begin{equation}\label{eq:M_instance}
  \mathcal{M}(\mathbf{h}_i^{(1)}, \mathbf{h}_i^{(2)}, \cdots, \mathbf{h}_i^{(V)}) = \frac{1}{V} \sum_{v=1}^V \mathbf{h}_i^{(v)}\mathbf{W}_v,
\end{equation}
where $\mathbf{W}_v$ is the $v$-th linear weights of mapping $\mathbf{h}_v$ into a consensus subspace.
After obtaining the concatenated representation $\mathbf{h}$ in Eq. (\ref{eq:latent_rep_v}), a classification network is employed to compute the label 
\begin{equation}
  \mathbf{y}_i' = g(\mathbf{h}_i)
\end{equation}
To measure the classification loss of whole networks, we adopt the widely used cross entropy loss as 
\begin{equation}\label{eq:loss}
  loss = - \sum_{i=1}^n \sum_{k=1}^K\mathbf{y}_{i,k}\log(\mathbf{y}'_{i,k}),
\end{equation}
where $\mathbf{y}_i$ is the one-hot code of ground truth $\mathbf{Y}_i$ and $\mathbf{y}_{i,k}$ refers to the $k$-th element of $\mathbf{y}_i$.
Nevertheless, Stochastic Gradient Descent (SGD) is employed to minimize Eq. (\ref{eq:loss}).

\section{Experiment}\label{sec:exp}

\begin{figure*}[t] 
  \centering
  \includegraphics[width=0.98\linewidth]{fig/fig1-crop} \\
  \includegraphics[width=0.98\linewidth]{fig/fig2-crop}
  \caption{Packet feature distributions of \emph{Normal}, \emph{Adw}, \emph{Drp}, \emph{Rtk} and \emph{Susp} traffic.}
  \label{fig:packet}
\end{figure*}

\begin{figure*}[] 
  \centering
  \includegraphics[width=0.95\linewidth]{fig/fig3-crop} \\
  \includegraphics[width=0.95\linewidth]{fig/fig4-crop}
  % \includegraphics[width=0.95\linewidth]{fig/fig5-crop}
  \caption{TLS feature distributions of \emph{Normal}, \emph{Adw}, \emph{Drp}, \emph{Rtk} and \emph{Susp} traffic. Note the distribution of \emph{Selected Extensions} is not shown due to space limit.}
  \label{fig:tls}
\end{figure*}
\begin{figure*}[] 
  \centering
  \includegraphics[width=0.95\linewidth]{fig/fig6-crop} \\
  \includegraphics[width=0.95\linewidth]{fig/fig7-crop} \\
  \includegraphics[width=0.95\linewidth]{fig/fig8-crop}
  \caption{Certificate feature distributions of \emph{Normal}, \emph{Adw}, \emph{Drp}, \emph{Rtk} and \emph{Susp} traffic.}
  \label{fig:cert}
\end{figure*}

\subsection{Setting}

In order to validate effectiveness and superiority of the proposed method, we conduct extensive experiments on public datasets of Malware Capture Facility Project (MCFP) \cite{stratodatasets}.  
They are collected by continually feeding intrusion prevention system with malware and normal traffic.
We adopt the provided \emph{.pcap} files with all payload data.
Nevertheless, we upload the corresponding malware executable files to \emph{VirusTotal}\footnote[8]{\url{https://www.virustotal.com}} which gathers detection results from a cluster of security companies.
In this paper, the results of \emph{Avast} are used to label the traffic flows.
In data cleaning, we remove the traffic of the minority classes and keep the majority, including \emph{Normal}, \emph{Adw}, \emph{Drp}, \emph{Rtk} and \emph{Susp}.
As a result, 520354 traffic flows are left, and their details are specified in Table \ref{tab:traffic_detail}.
Moreover, $80\%$ flows are randomly selected as training data, while the others used in test.

Three typical solutions are also employed in the following experiments, i.e. Support Vector Machine (SVM) \cite{DBLP:journals/sac/SmolaS04}, $k$-Nearest Neighbors (kNN) \cite{DBLP:journals/tnn/ShakhnarovichDI08} and Fully-connected Neural Networks (NN).
Since they operate on single-view data, we concatenate the three sets of extracted features into a unified vector and feed them directly.
For ease of expression, the proposed method is abbreviated as EMTD.
Furthermore, we invoke \emph{scikit-learn}\footnote[9]{\url{https://scikit-learn.org}} package to reproduce SVM and kNN methods, while NN and EMTD are implemented by ourself and released at \emph{Github}\footnote[10]{\emph{The URL will be available after review.}}.

Denoting malware traffic with $1$ and the normal with $0$, corresponding confusion matrix can be defined in Table \ref{tab:confusion_matrix}.
\begin{table}[h]
\centering
\caption{Confusion matrix definition.}
\small
\renewcommand\arraystretch{1.5}
\label{tab:confusion_matrix}
\begin{tabular}{|l|l|l|}
\hline
 & Detection: 1 & Detection: 0 \\ \hline
Label: 1 & True Positive (TP) & False Negative (FN) \\ \hline
Label: 0 & False Positive (FP) & True Negative (TN) \\ \hline
\end{tabular}
\end{table}
On the basis, we employ three metrics to measure detection performance, including
\begin{enumerate}
  \item \emph{FPR}: The rate of normal traffic flows misclassified as malware generated, as shown in Eq. (\ref{eq:fpr});
  \begin{equation}\label{eq:fpr}
    \text{FPR} = \frac{\text{FP}}{\text{FP} + \text{TN}}
  \end{equation}
  \item \emph{FNR}: The rate of malware traffic flows misclassified as normal, as shown in Eq. (\ref{eq:fnr});
  \begin{equation}\label{eq:fnr}
    \text{FNR} = \frac{\text{FN}}{\text{FN} + \text{TP}}
  \end{equation}
  \item \emph{Error Rate}: The rate of misclassified traffic flows, as shown in Eq. (\ref{eq:error_rate}).
  \begin{equation}\label{eq:error_rate}
    \text{Error Rate} = \frac{\text{FP} + \text{FN}}{\text{FP} + \text{TN} + \text{FN} + \text{TP}}
  \end{equation}
\end{enumerate}

\subsection{Feature insight}

Machine learning methods, including the proposed method, detect anomalies via discovering the statistical differences among classes.
Therefore, we visualize feature distributions of the three sets in Fig. \ref{fig:packet}, \ref{fig:tls} and \ref{fig:cert}, so as to provide an insight of detection basis.
It can be observed that normal traffic exhibits its distinctions over malware traffic of the considered four types, i.e. \emph{Adw}, \emph{Drp}, \emph{Rtk} and \emph{Susp}.
Taking feature \emph{Inbound Bytes} for an instance, the byte number of normal traffic scatters dispersedly, with about $5\%$ bigger than $6k$, which differs from the others obviously.
Meanwhile, \emph{Adw} traffic byte number distributes mostly in the range from $2k$ to $4k$, together with around $20\%$ smaller than $2k$ or from $4k$ to $6k$.
As a comparison, all byte numbers of \emph{Drp} and \emph{Rtk} are located in the range from $2k$ to $4k$.
Nevertheless, almost all byte numbers of \emph{Susp} are smaller than $2k$ or from $4k$ to $6k$.
Similar observations can be obtained in not only packet features but also TLS features and certificate features.
Some other phenomenons are also observed as follows:
\begin{enumerate}
  \item Normal and malware traffic shows no difference on a small proportion of features, including \emph{Offered Compression Method}, \emph{Selected Compression Method} and \emph{Bad Certificate Number}, which seems nonessential in the proposed method.
  However, this may be caused by limitation of the used dataset that not all distributions of traffic are collected.
  Therefore, we also keep them in the feature list. 
  \item Although certain feature values correspond to a minor percent of traffic, they are crucial in detecting anomalies.
  For example, only around $5\%$ of normal traffic carry more than $6k$ inbound bytes, as shown in the first subplot of Fig. (\ref{fig:packet}). 
  As a comparison, none of malware traffic does so, exhibiting the most difference for detection.
\end{enumerate}
To sum up, the obvious distribution differences among all traffic classes provide the fundamental basis of detecting malware traffic with machine learning techniques.

\begin{table}[t]
\centering
\caption{Comparison of malware traffic detection results.}
\small
\label{tab:detection_result}
\begin{tabular}{llcccc}
\toprule
Metric ($\%$) & Setting & \multicolumn{1}{c}{SVM} & \multicolumn{1}{c}{kNN} & \multicolumn{1}{c}{NN} & \multicolumn{1}{c}{EMTD} \\ \midrule
\multirow{5}{*}{FPR} & Adw$^+$ & 0.0075 & 0.0075 & 0.0150 & \textbf{0.0075} \\
 & Drp$^+$ & 0.0000 & 0.0150 & 0.0000 & \textbf{0.0000} \\
 & Rtk$^+$ & 0.0000 & 0.0000 & 0.0000 & \textbf{0.0000} \\
 & Susp$^+$ & 0.0000 & 0.0000 & 0.0300 & \textbf{0.0000} \\
 & All & \textbf{0.0150} & 0.0525 & 0.0525 & 0.0450 \\ \midrule
\multirow{5}{*}{FNR} & Adw$^+$ & 0.0504 & 0.0504 & 0.0504 & \textbf{0.0336} \\
 & Drp$^+$ & 0.0112 & \textbf{0.0070} & 0.0112 & 0.0112 \\
 & Rtk$^+$ & 0.0000 & 0.1144 & 0.0000 & \textbf{0.0000} \\
 & Susp$^+$ & 0.0427 & 0.0285 & 0.0000 & \textbf{0.0000} \\
 & All & 0.0220 & 0.0694 & 0.0143 & \textbf{0.0132} \\ \midrule
\multirow{5}{*}{Error Rate} & Adw$^+$ & 0.0207 & 0.0207 & 0.0259 & \textbf{0.0156} \\
 & Drp$^+$ & 0.0094 & \textbf{0.0082} & 0.0094 & 0.0094 \\
 & Rtk$^+$ & 0.0000 & 0.0360 & 0.0000 & \textbf{0.0000} \\
 & Susp$^+$ & 0.0147 & 0.0098 & 0.0197 & \textbf{0.0000} \\
 & All & 0.0211 & 0.0673 & 0.0192 & \textbf{0.0173} \\ \bottomrule
\end{tabular}
\end{table}


\subsection{Detection result}\label{sec:detection_result}

To demonstrate effectiveness of the proposed multi-view algorithm on malware traffic detection, we compare it with three basic classification algorithms, i.e. SVM, kNN and NN.
Corresponding results are presented in Table \ref{tab:detection_result}.
It is worth to note that \emph{Adw$^+$} refers to mixing \emph{Adw} traffic with \emph{Normal} traffic.
Obviously, \emph{Drp$^+$}, \emph{Rtk$^+$} and \emph{Susp$^+$} follow the same setting.
In addition, \emph{All} represents mixing all traffic together and regarding all classes of malware traffic as anomaly.
Nevertheless, we mark EMTD's results in boldface, if they are the smallest. 
Otherwise, the smallest results are marked.
From the results shown in Table \ref{tab:detection_result}, we obtain the following observations:
\begin{enumerate}
  \item All methods exhibits lower than $0.1\%$ FPR, FNR and Error Rate on all settings.
  It is even better that the metrics are smaller than $0.0001\%$ in a part of settings.
  This well validates effectiveness of the extracted features.
  \item Although the proposed multi-view algorithm presents little worse than SVM in \emph{All} setting on FPR and kNN in \emph{Drp$^+$} setting on FNR and Error Rate, it achieves the least FPR, FNR and Error Rate in almost all settings.
  This sufficiently illustrates superiority of the proposed multi-view method over widely used approaches.
\end{enumerate}
Overall, effectiveness of the extracted features and superiority of the proposed multi-view method are both validated experimentally. 

\begin{table}[t]
\centering
\caption{Detection results on different feature set.}
\small
\label{tab:abalation_study}
\begin{tabular}{llcccc}
\toprule
Metric ($\%$) & Setting & \multicolumn{1}{c}{Packet} & \multicolumn{1}{c}{TLS} & \multicolumn{1}{c}{Cert.} & \multicolumn{1}{c}{All$^*$} \\ \midrule
\multirow{5}{*}{FPR} & Adw$^+$ & 0.0600 & 0.0450 & 2.1310 & \textbf{0.0075} \\
 & Drp$^+$ & 0.8254 & 0.0075 & 0.9905 & \textbf{0.0000} \\
 & Rtk$^+$ & 0.6903 & 0.0150 & 0.9905 & \textbf{0.0000} \\
 & Susp$^+$ & 5.2750 & 0.0375 & 3.2040 & \textbf{0.0000} \\
 & All & 1.3807 & 0.0525 & 4.1044 & \textbf{0.0450} \\ \midrule
\multirow{5}{*}{FNR} & Adw$^+$ & 0.1008 & 0.0336 & 8.9535 & \textbf{0.0336} \\
 & Drp$^+$ & 0.0335 & 0.0112 & \textbf{0.0000} & 0.0112 \\
 & Rtk$^+$ & 4.5105 & 5.2296 & 0.0000 & \textbf{0.0000} \\
 & Susp$^+$ & 0.6549 & 0.0285 & 23.8326 & \textbf{0.0000} \\
 & All & 0.5576 & 0.3692 & 2.4464 & \textbf{0.0132} \\ \midrule
\multirow{5}{*}{Error Rate} & Adw$^+$ & 0.0726 & 0.0415 & 4.2376 & \textbf{0.0156} \\
 & Drp$^+$ & 0.1577 & 0.0106 & 0.1553 & \textbf{0.0094} \\
 & Rtk$^+$ & 1.8924 & 1.6559 & 0.6788 & \textbf{0.0000} \\
 & Susp$^+$ & 3.6804 & 0.0344 & 10.3238 & \textbf{0.0000} \\
 & All & 0.6630 & 0.3286 & 2.6588 & \textbf{0.0173} \\ \bottomrule
\end{tabular}
\end{table}

\subsection{Ablation study and convergence}

An extra experiment is also conducted to verify EMTD's capability of integrating information from multiple data views.
By following the same settings in Section \ref{sec:detection_result}, we separately feed the proposed multi-view method with each set of features, and compare their results with feeding all sets of features at once.
Corresponding results are obtained in Table \ref{tab:abalation_study}.
Note that \emph{Cert.} is the abbreviation of \emph{Certificate}, and \emph{All$^*$} refers to the combination of three feature sets. 
We can see that FPR, FNR and Error Rate decrease dramatically when using all features, well demonstrating that the proposed multi-view method can integrate the beneficial details of packet, TLS and certificate features.

In addition, loss, FPR, FNR and Error Rate of test data are recorded by training epoch and shown in Fig. \ref{fig:convergence}.
The subplot corresponding to \emph{All} setting is omitted due to space limit.
It can be observed that 
\begin{enumerate}
  \item The target loss monotonically decreases to a minimum, verifying that training and test data are drawn from a single distribution and minimizing loss of training data will result in a small loss value of test data.
  \item In \emph{Adw$^+$}, \emph{Rtk$^+$} and \emph{Susp$^+$} settings, FNR and Error Rate drop sharply at first, then decrease slowly with minor fluctuations towards to constants.
  This illustrates the model performance is improved along with training process and loss decrease, hence proves effectiveness of the designed loss function.
  \item In \emph{Adw$^+$}, \emph{Rtk$^+$} and \emph{Susp$^+$} settings, FPR starts at zero, which is caused by the model initially treats all data as normal. 
  \item In \emph{Drp$^+$} setting, FPR, FNR and Error Rate start at small values, then drop to a minimum with less than $5$ epochs, and keep stable during the latter training. 
  This may be caused by the high separability between \emph{Normal} and \emph{Drp} traffic. 
\end{enumerate}
Overall, loss and all metrics decrease along with training epoch, demonstrating effectiveness of the designed loss function.

\begin{figure*}[t] 
  \centering
  \includegraphics[width=\linewidth]{fig/convergence_four-crop}
  \caption{Loss, FPR, FNR and Error Rate variation by training epoch in \emph{Adw$^+$}, \emph{Drp$^+$}, \emph{Rtk$^+$} and \emph{Susp$^+$} settings.}
  \label{fig:convergence}
\end{figure*}

\section{Discussion}

% The proposed approach of malware traffic detection enjoys the following merits:
We distinguish the proposed approach of malware traffic detection from three aspects in the following.

1) Strength.
DPI-tailored methods break the TLS protocol's end-to-end guarantee (resulting in privacy leakage threat) and can severely worsen network transmission latency.
In contrast, the proposed approach extracts traffic flow features and trains a machine learning model for detection, thus mitigating the problems above.
In addition, the proposed method extracts features from the TCP and TLS building process, making it possible to detect malware traffic before malicious behaviors happen.

2) Novelty.
Existing learning-based methods concatenate all features into a universal vector. 
In contrast, we propose a purposive multi-view learning neural network that optimally integrates the three sets of features.
To the best of our knowledge, this is the first work of applying a multi-view learning model in malware traffic detection. It may provide beneficial guidance for follow-up research into this direction.

3) Extension.
The proposed method improves the detection performance in two aspects: feature engineering and model design.
In addition, the features are not limited to the extracted three sets of features.
(For example, DNS, URL, and other features can be utilized in future work.) 

\section{Conclusion}

This paper proposes a new multi-view approach (entitled EMTD) to detect TLS encrypted malware traffic.
It improves the detection performance in both feature engineering and machine learning designs.
To the best of our knowledge, this is the first work of applying multi-view learning in malware traffic detection.
Nevertheless, we conduct extensive experiments on the MCFP dataset, verifying its effectiveness and superiority over classical methods.

%%@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
% \acknowledgements{
% }
%%@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
%\bibliographystyle{EMSart}
\bibliographystyle{cjereport}
\bibliography{refs}

\iffalse
%%============================================
% \authorinfomation{Firstname1 Middlename1 Lastname1}{Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx}

% \authorinfomation{Firstname2 Middlename2 Lastname2}{Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx}

% \authorinfomation{Firstname3 Middlename3 Lastname3}{Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx}

% \authorinfomation{Firstname4 Middlename4 Lastname4}{Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx}

% \authorinfomation{Firstname5 Middlename5 Lastname5}{Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx}

% \authorinfomation{Firstname6 Middlename6 Lastname6}{Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx}

% \authorinfomation{Firstname7 Middlename7 Lastname7}{Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx Xxxx xxx xxx xxxx xxx}

\fi
%%@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
%\vspace{-2em}
%% 隐藏作者信息（如需展示，请注释掉 \iffalse 与最后的 \fi ）


%% 隐藏作者信息

\end{document}
